Managing caching of extents of tracks in a first cache, second cache and storage

ABSTRACT

Provided are a computer program product, system, and method for managing caching of extents of tracks in a first cache, second cache and storage device. A determination is made of an eligible track in a first cache eligible for demotion to a second cache, wherein the tracks are stored in extents configured in a storage device, wherein each extent is comprised of a plurality of tracks. A determination is made of an extent including the eligible track and whether second cache caching for the determined extent is enabled or disabled. The eligible track is demoted from the first cache to the second cache in response to determining that the second cache caching for the determined extent is enabled. Selection is made not to demote the eligible track in response to determining that the second cache caching for the determined extent is disabled.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a computer program product, system, andmethod for managing caching of extents of tracks in a first cache,second cache and storage.

2. Description of the Related Art

A cache management system buffers tracks in a storage device recentlyaccessed as a result of read and write operations in a faster accessstorage device, such as memory, than the storage device storing therequested tracks. Subsequent read requests to tracks in the fasteraccess cache memory are returned at a faster rate than returning therequested tracks from the slower access storage, thus reducing readlatency. The cache management system may also return complete to a writerequest when the modified track directed to the storage device iswritten to the cache memory and before the modified track is written outto the storage device, such as a hard disk drive. The write latency tothe storage device is typically significantly longer than the latency towrite to a cache memory. Thus, using cache also reduces write latency.

A cache management system may maintain a linked list having one entryfor each track stored in the cache, which may comprise write databuffered in cache before writing to the storage device or read data. Inthe commonly used Least Recently Used (LRU) cache technique, if a trackin the cache is accessed, i.e., a cache “hit”, then the entry in the LRUlist for the accessed track is moved to a Most Recently Used (MRU) endof the list. If the requested track is not in the cache, i.e., a cachemiss, then the track in the cache whose entry is at the LRU end of thelist may be removed (or destaged back to storage) and an entry for thetrack data staged into cache from the storage is added to the MRU end ofthe LRU list. With this LRU cache technique, tracks that are morefrequently accessed are likely to remain in cache, while data lessfrequently accessed will more likely be removed from the LRU end of thelist to make room in cache for newly accessed tracks.

The Easy Tier application offered by International Business MachinesCorporation (“IBM”) migrates extents of frequently accessed data fromhard disk drive storage to a solid state storage device (SSD), which hasfaster access than the disk drives. The Easy Tier application monitorsInput/Output (I/O) workload to extents of tracks, and if the workload oractivity with respect to the extent reaches a certain threshold, thenthe Easy Tier application migrates the extent to the SSD where accesstimes are improved. In this way, “hot extents”, those frequentlyaccessed, are migrated to the SSD, while “cold extents”, those deemedless frequently accessed, are migrated from the SSD to the hard diskdrives.

There is a need in the art for improved techniques for using cache in astorage system.

SUMMARY

Provided are a computer program product, system, and method for managingcaching of extents of tracks in a first cache, second cache and storagedevice. A determination is made of an eligible track in a first cacheeligible for demotion to a second cache, wherein the tracks are storedin extents configured in a storage device, wherein each extent iscomprised of a plurality of tracks. A determination is made of an extentincluding the eligible track and whether second cache caching for thedetermined extent is enabled or disabled. The eligible track is demotedfrom the first cache to the second cache in response to determining thatthe second cache caching for the determined extent is enabled. Selectionis made not to demote the eligible track in response to determining thatthe second cache caching for the determined extent is disabled.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an embodiment of a computing environment.

FIG. 2 illustrates an embodiment of first cache management information.

FIG. 3 illustrates an embodiment of second cache management information.

FIG. 4 illustrates an embodiment of a first cache control block.

FIG. 5 illustrates an embodiment of a second cache control block.

FIG. 6 illustrates an embodiment of a spatial index entry.

FIG. 7 illustrates an embodiment of an extent caching information entry.

FIG. 8 illustrates an embodiment of an extent activity entry.

FIG. 9 illustrates an embodiment of operations to demote unmodifiedtracks from the first cache.

FIG. 10 illustrates an embodiment of operations to destage modifiedtracks from the first cache.

FIG. 11 illustrates an embodiment of operations to add a track to thefirst cache.

FIG. 12 illustrates an embodiment of operations to promote a track tothe second cache.

FIG. 13 illustrates an embodiment of operations to free space in thesecond cache.

FIG. 14 illustrates an embodiment of operations to destage modifiedtracks from the second cache.

FIG. 15 illustrates an embodiment of operations to process a request fortracks to return to a read request.

FIG. 16 illustrates an embodiment of operations to monitor destage/stageactivity with respect to extents.

FIG. 17 illustrates an embodiment of operations to migrate an extent inthe storage device.

FIG. 18 illustrates an embodiment of operations to demote a track fromthe first cache to the second cache.

FIG. 19 illustrates an embodiment of operations to manage the secondcache caching setting.

FIG. 20 illustrates an embodiment of operations to initiate migrationwhen the second cache only has unmodified tracks.

DETAILED DESCRIPTION

Described embodiments provide techniques to manage the migration ofextents of tracks in a storage device to a second cache that provides asecondary cache to a first cache, where host read and write activity isdirected to the first cache. The second cache stores tracks beingtransferred between the first cache and the storage device, where thefirst cache may comprise a faster access device than the second cacheand the second cache may comprise a faster access device than thestorage device. With described embodiments, tracks are demoted from thefirst cache to the second cache if second cache caching is enabled forthe extent. The second cache caching for an extent in the second cacheis disabled if an activity level of destage and stage operations for theextent exceed an activity threshold and if a distribution of the tracksin the extent subject to destage and stage operations exceeds adistribution threshold. This prevents the first cache from demoting aneligible track to the second cache that is in an extent already in thesecond cache that has a relatively high amount of activity anddistribution of activity. Caching to the tracks in the extent in thesecond cache may be enabled after a predetermined time or after theextent is migrated from the second cache back to the storage.

FIG. 1 illustrates an embodiment of a computing environment. A pluralityof hosts 2 a, 2 b . . . 2 n may submit Input/Output (I/O) requests to astorage controller 4 over a network 6 to access data at volumes 8 (e.g.,Logical Unit Numbers, Logical Devices, Logical Subsystems, etc.) in astorage 10. The storage controller 4 includes a processor complex 12,including one or more processors with single or multiple cores, a firstcache 14, a first cache backup device 16, to backup tracks in the cache14, and a second cache 18. The first 14 and second 18 caches cache datatransferred between the hosts 2 a, 2 b . . . 2 n and the storage 10. Thefirst cache backup device 16 may provide non-volatile storage of tracksin the first cache 14. In a further embodiment, the first cache backupdevice 16 may be located in a cluster or hardware on a different powerboundary than that of the first cache 14.

The storage controller 4 has a memory 20 that includes a storage manager22 for managing the transfer of tracks transferred between the hosts 2a, 2 b . . . 2 n and the storage 10 and a cache manager 24 that managesdata transferred between the hosts 2 a, 2 b . . . 2 n and the storage 10in the first cache 14, first cache backup device 16, and the secondcache 18. A track may comprise any unit of data configured in thestorage 10, such as a track, Logical Block Address (LBA), etc., which ispart of a larger grouping of tracks, such as a volume, logical device,etc. The cache manager 24 maintains first cache management information26 and second cache management information 28 to manage read(unmodified) and write (modified) tracks in the first cache 14 and thesecond cache 18. A first cache backup device index 30 provides an indexof track identifiers to a location in the first cache backup device 16.

The storage manager 22 may configure extents in the storage 10, whereeach extent is assigned a plurality of tracks in the storage 10. In thisway, the storage 10 manages tracks in groups of extents, whereas thecache manager 24 manages tracks in the first 14 and second 18 caches.

The second cache 18 may store tracks in a log structured array (LSA) 32,where tracks are written in a sequential order as received, thusproviding a temporal ordering of the tracks written to the second cache18. In a LSA, later versions of tracks already present in the LSA arewritten at the end of the LSA 32. In alternative embodiments, the secondcache 18 may store data in formats other than in an LSA.

The storage controller 4 may further include a migration manager 34 thatmanages the migration of extents between the storage 10 and the secondcache 18. In certain embodiments, the migration manager 34 may migrateextents in the storage 10 to the second cache 18 that have a high levelof activity, such as a high level of being subject to destage and stageoperations. The migration manager 34 may be part of the storage manager22 or a separate program.

The storage manager 22, cache manager 24, and migration manager 34 areshown in FIG. 1 as program code loaded into the memory 20 and executedby the processor complex 12. Alternatively, some or all of the functionsmay be implemented in hardware devices in the storage controller 4, suchas in Application Specific Integrated Circuits (ASICs).

In one embodiment, the first cache 14 may comprise a Random AccessMemory (RAM), such as a Dynamic Random Access Memory (DRAM), and thesecond cache 18 may comprise a flash memory, such as a solid statedevice, and the storage 10 is comprised of one or more sequential accessstorage devices, such as hard disk drives and magnetic tape. The storage10 may comprise a single sequential access storage device or maycomprise an array of storage devices, such as a Just a Bunch of Disks(JBOD), Direct Access Storage Device (DASD), Redundant Array ofIndependent Disks (RAID) array, virtualization device, etc. In oneembodiment, the first cache 14 is a faster access device than the secondcache 18, and the second cache 18 is a faster access device than thestorage 10. Further, the first cache 14 may have a greater cost per unitof storage than the second cache 18 and the second cache 18 may have agreater cost per unit of storage than storage devices in the storage 10.

The first cache 14 may be part of the memory 20 or implemented in aseparate memory device, such as a DRAM. In one embodiment, the firstcache backup device 16 may comprise a non-volatile backup storage (NVS),such as a non-volatile memory, e.g., battery backed-up Random AccessMemory (RAM), static RAM (SRAM), etc.

The network 6 may comprise a Storage Area Network (SAN), a Local AreaNetwork (LAN), a Wide Area Network (WAN), the Internet, and Intranet,etc.

FIG. 2 illustrates an embodiment of the first cache managementinformation 26 including a track index 50 providing an index of tracksin the first cache 14 to control blocks in a control block directory 52;an unmodified sequential LRU list 54 providing a temporal ordering ofunmodified sequential tracks in the first cache 14; a modified LRU list56 providing a temporal ordering of modified sequential andnon-sequential tracks in the first cache 14; and an unmodifiednon-sequential LRU list 58 providing a temporal ordering of unmodifiednon-sequential tracks in the first cache 14.

In certain embodiments, upon determining that the first cache backupdevice 16 is full, the modified LRU list 56 is used to destage modifiedtracks from the first cache 14 so that the copy of those tracks in thefirst cache backup device 16 may be discarded to make room in the firstcache backup device 16 for new modified tracks.

FIG. 3 illustrates an embodiment of the second cache managementinformation 28 including a track index 70 providing an index of tracksin the second cache 18 to control blocks in a control block directory72; an unmodified list 74 providing a temporal ordering of unmodifiedtracks in the second cache 18; a spatial index 76 providing a spatialordering of the modified tracks in the second cache 18 based on thephysical locations in the storage 10 at which the modified tracks arestored; extent caching information 80 providing information on extentsin the second cache 18 that have been migrated from the storage 10; andextent activity 80 providing information on read/stage and write/destageactivity with respect to extents in the second cache 18 or the storage10.

All the LRU lists 54, 56, 58, and 74 may include the track IDs of tracksin the first cache 14 and the second cache 18 ordered according to whenthe identified track was last accessed. The LRU lists 54, 56, 58, and 74have a most recently used (MRU) end indicating a most recently accessedtrack and a LRU end indicating a least recently used or accessed track.The track IDs of tracks added to the caches 14 and 18 are added to theMRU end of the LRU list and tracks demoted from the caches 14 and 18 areaccessed from the LRU end. The track indexes 50 and 70 and spatial index76 may comprise a scatter index table (SIT). Alternative type datastructures may be used to provide the temporal ordering of tracks in thecaches 14 and 18 and spatial ordering of tracks in the second cache 18.

Non-sequential tracks may comprise Online Line Transaction Processing(OLTP) tracks, which often comprise small block writes that are notfully random and have some locality of reference, i.e., have aprobability of being repeatedly accessed.

FIG. 4 illustrates an embodiment of a first cache control block 100entry in the control block directory 52, including a control blockidentifier (ID) 102, a first cache location 104 of the physical locationof the track in the first cache 14, information 106 indicating whetherthe track is modified or unmodified, and information 108 indicatingwhether the track is a sequential or non-sequential access.

FIG. 5 illustrates an embodiment of a second cache control block 120entry in the second cache control block directory 72, including acontrol block identifier (ID) 122, an LSA location 124 where the trackis located in the LSA 32, and information 126 indicating whether thetrack is modified or unmodified.

FIG. 6 illustrates a spatial index entry 140 including a trackidentifier 142 of a track in the second cache 18 and the physicallocation 144 of where the track is stored in the storage 10, such as acylinder, platter, block address, and storage device identifier.

FIG. 7 illustrates an embodiment of an instance of extent cachinginformation entry 150 included in the extent caching information 78 foreach extent in the second cache 18 indicating an extent identifier (ID)152 of the extent and a second cache caching setting 154 indicatingwhether second cache caching is enabled or disabled for the extent 152in the second cache 18. If second cache caching 154 is disabled for anextent 152 in the second cache 18, then tracks in that extent 152 cannotbe demoted form the first cache 14 to the second cache 18. Further, ifsecond cache caching 154 is disabled, then tracks in the extent 152 inthe second 18 cache may not be eligible for demotion from the secondcache 18 to the storage 10.

FIG. 8 illustrates an embodiment of an extent activity entry 160included in the extent activity 80 information for each extentindicating an extent identifier (ID) 162 of the extent, a destage/writeactivity 164 and stage/read activity 166 indicating a number ofdestage/writes and stage/reads, respectively, to tracks in the extentwhile in the storage 10 or the second cache 18; and an activitydistribution 168 indicating a distribution of the destage/stage activitywith respect to different tracks of the extent. This distribution value168 may indicate a number or proportion of tracks subject to activity, astandard deviation of tracks subject to activity, or some otherindicator of the number of tracks participating in the destage/stageactivity.

In one embodiment, the extent activity 164 and 166 may indicate thenumber of times a track in an extent located in the storage 10 or secondcache 18 is written/destaged or read/staged from the first cache 14. Theactivity may not reflect the number of times the track is read orwritten while in the first cache 14, only the activity with respect tomoving an extent track from the first cache 14 to the second cache 18 orstorage 10.

FIG. 9 illustrates an embodiment of operations performed by the cachemanager 24 to demote unmodified tracks from the first cache 14. Thedemote operation may be initiated upon determining to free space in thefirst cache 14. Upon initiating (at block 200) an operation to determinewhether to remove tracks from the first cache 14 to free space in thefirst cache 14, the cache manager 24 determines (at block 202) whetherto demote non-sequential or sequential unmodified tracks based onexpected hits to different types of unmodified tracks. If (at block 204)the determination is to demote unmodified sequential tracks, then thecache manager 24 uses (at block 206) the unmodified sequential LRU list54 to determine unmodified sequential tracks to demote, from the LRU endof the list, which are not promoted to the second cache 18. If (at block204) the determination is made to demote unmodified non-sequentialtracks, then the cache manager uses the unmodified non-sequential LRUlist 58 to determine (at block 208) unmodified non-sequential tracks todemote. The unmodified non-sequential tracks are promoted (at block 210)to the second cache 18.

FIG. 10 illustrates an embodiment of operations performed by the cachemanager 24 to destage modified tracks from the first cache 14. The cachemanager 24 may regularly destage tracks as part of scheduled operationsand increase the rate of destages if space is needed in the first cachebackup device 16. Upon initiating (at block 250) the operation todestage modified tracks, the cache manager 24 processes (at bock 252)the modified LRU list 56 to determine modified tracks to destage, fromthe LRU end of the LRU list 56. If (at block 254) the determinedmodified tracks are sequential, then the cache manager 24 writes (atblock 256) the determined modified sequential tracks to the storage 10,bypassing the second cache 18. If (at block 254) the modified tracks arenon-sequential, then the cache manager 24 promotes (at block 258) thedetermined modified non-sequential tracks to the second cache 18 anddiscards (at block 260) the copy of the determined modified tracks fromthe first cache backup device 16.

With the operations of FIGS. 9 and 10, non-sequential tracks are demotedbut not promoted to the second cache 18. Sequential modified (writes)are written directly to the storage 10, bypassing the second cache.Sequential unmodified tracks (reads) are discarded and not copiedelsewhere, and unmodified non-sequential tracks demoted from the firstcache 14 are promoted to the second cache 18.

FIG. 11 illustrates an embodiment of operations performed by the cachemanager 24 to add, i.e., promote, a track to the first cache 14, whichtrack may comprise a write or modified track from a host 2 a, 2 b . . .2 n, a non-sequential track in the second cache 18 that is subject to aread request and as a result moved to the first cache 14, or readrequested data not found in either cache 14 or 18 and retrieved from thestorage 10. Upon receiving (at block 300) the track to add to the firstcache 14, the cache manager 24 creates (at block 301) a control block100 (FIG. 4) for the track to add indicating the 104 location in thefirst cache 14 and whether the track is modified/unmodified 106 andsequential/non-sequential 108. This control block 100 is added to thecontrol block directory 52 of the first cache 14. The cache manager 24adds (at block 302) an entry to the first cache track index 50 havingthe track ID of track to add and an index to the created cache controlblock 100 in the control block directory 52. An entry is added (at block304) to the MRU end of the LRU list 54, 56 or 58 of the track type ofthe track to add. If (at block 306) the track to add is a modifiednon-sequential track, then the track to add is also copied (at block308) to the first cache backup device 16 and an entry is added to thefirst cache backup device index 30 for the added track. If (at block306) the track to add is unmodified sequential, control ends.

FIG. 12 illustrates an embodiment of operations performed by the cachemanager 24 to promote non-sequential tracks to the second cache 18 thatare being demoted from the first cache 14. Upon initiating (at block350) the operation to promote a track to the second cache 18, the cachemanager 24 adds (at block 352) the track being promoted to the LSA 32 inthe second cache 18 and creates (at block 354) a control block 120 (FIG.5) for the track to add indicating the track location 124 in the LSA 32and whether the track is modified/unmodified 126. An entry is added (atblock 356) to the second cache track index 70 having the track ID of thepromoted track and an index to the created cache control block 120 inthe control block directory 72 for the second cache 18. If (from the nobranch of block 358) the track being promoted is unmodified data, thenthe cache manager 24 indicates (at block 360) the promoted track at theMRU end of the unmodified LRU list 74, such as by adding the track ID tothe MRU end. If (at block 358) the promoted track is modified data, thenthe cache manager 24 determines (at block 362) a physical location ofthe where to write the modified promoted track on the storage 10 andadds (at block 364) an entry to the spatial index 76 indicating thetrack ID 142 of the promoted track and the determined physical location144 of the promoted track on the storage 10.

FIG. 13 illustrates an embodiment of operations performed by the cachemanager 24 to free space in the second cache 18 for new tracks to add tothe second cache 18, i.e., tracks being demoted from the first cache 14or promoted from the storage 10. Upon initiating this operation (atblock 400) the cache manager 24 determines (at block 402) unmodifiedtracks in the second cache 18 from the LRU end of the unmodified LRUlist 74 and invalidates (at block 404) the determined unmodified trackswithout destaging the invalidated unmodified tracks to the storage 10,and also removes the invalidated unmodified tracks from the unmodifiedLRU list 74.

FIG. 14 illustrates an embodiment of operations performed by the cachemanager 24 to destage modified tracks in the second cache 12 to thestorage 10. Upon initiating (at block 420) the destage operation, thecache manager 24 uses (at block 422) the spatial index 76 to determinemodified tracks in the second cache 18 to group that are at proximatephysical locations on the storage device based on their order in thespatial index 76, such as a group of tracks at consecutive entries 140(FIG. 6) in the location ordered spatial index 76. The determinedgrouped modified tracks are destaged (at block 424) from the secondcache 18 to the storage device 10.

The operations of FIG. 14 optimize write operations to the storage 10,which comprises a sequential access storage device, by having thestorage 10 write tracks at physical locations at closest physicalproximity on the storage 10 writing surface, thus minimizing the amountof seeking and movement of the storage 10 write head to write the tracksin the second cache 18 to storage 10. The tracks were promoted to thesecond cache 18 from the from the first cache 16 based on temporalfactors, such as their position in an LRU list 54, 56, 58, and not in anorder optimized for sequential writing to a sequential access storage10, such as a hard disk drive. Thus, the operations of FIG. 12 optimizethe temporally ordered modified tracks based on their spatial locationon the storage 10 to optimize writing at the storage 10.

FIG. 15 illustrates an embodiment of operations performed by the cachemanager 24 to retrieve requested tracks for a read request from thecaches 14 and 18 and storage 10. The storage manager 22 processing theread request may submit requests to the cache manager 24 for therequested tracks. Upon receiving (at block 450) the request for thetracks, the cache manager 24 uses (at block 454) the first cache trackindex 50 to determine whether all of the requested tracks are in thefirst cache 14. If (at block 454) all requested tracks are not in thefirst cache 14, then the cache manager 24 uses (at block 456) the secondcache track index 70 to determine any of the requested tracks in thesecond cache 18 not in the first cache 14. If (at block 458) there areany requested tracks not found in the first 14 and second 18 caches,then the cache manager 24 determines (at block 460) any of the requestedtracks in the storage 10, from the second cache track index 70, not inthe first 14 and the second 18 caches. The cache manager 24 thenpromotes (at block 462) any of the determined tracks in the second cache18 and the storage 10 to the first cache 14. The cache manager 24 uses(at block 464) the first cache track index 50 to retrieve the requestedtracks from the first cache 14 to return to the read request. Theentries for the retrieved tracks are moved (at block 466) to the MRU endof the LRU list 54, 56, 58 including entries for the retrieved tracks.

With the operations of FIG. 15, the cache manager 24 retrieves requestedtracks from a highest level cache 14, then second cache 18 first beforegoing to the storage 10, because the caches 14 and 18 would have themost recent modified version of a requested track. The most recentversion is first found in the first cache 14, then the second cache 18if not in the first cache 14 and then the storage 10 if not in eithercache 14, 18.

FIG. 16 illustrates an embodiment of operations performed by the cachemanager 24 or other component, such as the storage manager 22 ormigration manager 34, to maintain the extent activity 82 information.Upon detecting (at block 500) an operation that destages or stages datawith respect to a track in an extent in the storage 10 or the secondcache 18, the cache manager 24 updates (at block 502) the destage/write164 or stage/read 166 activity information for the extent 162 to reflectthe additional activity. The cache manager 24 may further update theactivity distribution 168 information to indicate update a number of thetracks in the extent that have been subject to the stage or destageoperation. The distribution 168 information indicates the number oftracks of an extent involved in the activity.

In the operations of FIG. 16, the cache manager 24 tracks destage andstage activity when the track is destaged or staged between the firstcache 14 and the second cache 18, destaged or staged between the firstcache 14 and the storage 10, destaged or staged between the second cache18 and the storage 10, regardless of how many times the track ismodified or read while in the first cache 14 and second cache 18. Forinstance, a tracked destage/stage activity resulting in the update ofthe extent activity information 164, 166, 168 may comprise thepromotion/demotion of a track from the first cache 14 to the secondcache 18 at blocks 206 and 210 in FIG. 9 and block 258 in FIG. 10; thedestaging of a track from the first cache 14 to the storage 10 at block256 in FIG. 10; the promotion or staging for a track from the storage 10or second cache 18 to the first cache 14 at block 300 in FIG. 11; thepromotion/destaging of a track from the first cache 14 to the secondcache 18 at block 350 in FIG. 12; the destaging of modified tracks inthe second cache 18 to the storage 10 in FIG. 14; and thestaging/promotion of tracks in the second cache 18 or storage 10 to thefirst cache 14 at block 462 in FIG. 15.

FIG. 17 illustrates an embodiment of operations performed by themigration manager 34 to migrate an extent from the storage 10 to afaster access second cache 18. Upon (at block 520) initiating anoperation to determine whether to migrate an extent in the storage 10,the migration manager 34 determines (at block 522) activity (stage 164and/or destage 166 activity) at the extent in the storage device 10. Themigration manager 34 determines (at block 524) whether determinedactivity (such as total stage/read 164 and destage/write 166 activity)exceeds a threshold level of activity, such as a threshold number ofstage and destage operations. The measured values and threshold maycomprise different measurements, such as a number of measurements over atime period. If (at block 524) the threshold level of activity is notsatisfied, then control ends. Otherwise, if the threshold is met, thenthe migration manager 34 determines (at block 526) a distribution 168 ofthe tracks in the extents subject to destage and stage operations in thestorage device, i.e., the proportion, number of tracks, etc.,participating in the activity. If (at block 528) the determineddistribution of activity does not exceed a threshold distribution value,then control ends. Otherwise, if (at block 528) the distribution levelthreshold is satisfied, then the migration manager 34 migrates (at block530) the extent from the storage device 10 to the second cache 18 andindicates (at block 532) the second cache caching 154 for the extent 152as disabled, so that tracks in the extent will not promoted to ordemoted from the second cache 18.

In the described embodiments of FIG. 17, the migration manager 34 checksboth the amount of activity and distribution of activity. In alternativeembodiments, the migration manager 34 may check the threshold witheither the magnitude of activity or the distribution.

FIG. 18 illustrates an embodiment of operations performed by the cachemanager 24 to demote a track from the first cache 14 to the second cache18. Upon initiating (at block 550) an operation to demote a track fromthe first cache 14 to the second cache 18, the cache manager 14determines (at block 552) an eligible track in the first cache 14eligible for demotion to the second cache 18. A track may be determinedto be eligible for demotion according to FIGS. 9, 10, and 12. If (atblock 554) the track eligible to demote has modified data, then thecache manager 24 destages (at block 556) the modified data to the secondcache 18 so that the second cache 18 has the most current modifiedversion of the track. Otherwise, if (at block 554) the eligible track isunmodified, then the cache manager 24 determines (at block 558) anextent including the eligible track 558. If (at block 560) the secondcache caching 154 for the determined extent is not enabled, then thecache manager 24 selects (at block 562) not to demote the eligible trackand may discard the eligible track. Otherwise, if the second cachecaching 154 is disabled, then the cache manager 24 demotes (at block564) the eligible track from the first cache 14 to the second cache 18.

FIG. 19 illustrates an embodiment of operations performed by the cachemanager 24, or another component, e.g., 22, 34, to manage the secondcache caching 154 to determine whether to re-enable the second cachecaching 154 after it is disabled. Upon initiating (at block 600)management of the second cache caching 154 setting while disabled, thecache manager 24 may indicate (at block 602) the second cache caching154 for the extent as enabled after a predetermined time from when thesecond cache caching 154 for the extent was indicated as disabled.Further, while the second cache caching 154 is disabled, the cachemanager 24 may gather (at block 604) extent activity information 160 forthe extent and periodically determine (at block 606) the activity forthe extent while the extent is stored in the second cache 18. If (atblock 608) the periodically determined activity is greater than thethreshold level of activity, then the cache manager 24 indicates (atblock 610) the second cache caching 154 for the extent as disabledbecause the extent activity is still sufficiently high to warrantmaintaining the entire extent in the second cache 18. Otherwise, if (atblock 608) the determined activity for the extent has sufficientlyfallen, i.e., determined activity is less than the threshold, then thecache manage 24 or migration manager 34 may migrate (at block 612) theextent form the second cache 18 to the storage device 10.

In an alternative embodiment, the second cache caching 154 for theextent may be indicated as enabled in response to determining that theperiodically determined level of activity is less than the thresholdlevel of activity.

FIG. 20 illustrates an embodiment of operations performed by the cachemanager 24 and migration manager 34 to manage migration and second cachecaching settings 154 when the second cache 18 is a read-only cache. Uponinitiating (at block 630) migration operations, the migration manager 34performs the operations in FIG. 17. If (at block 634) the extent was notmigrated to the second cache 18 then control ends. Otherwise, if theextent was migrated to the second cache 18, then the cache manager 18determines (at block 636) a level of write/destage 164 activity and alevel of stage/read 166 activity to the tracks in the extent. If (atblock 638) the destage/write activity 164 is sufficiently high, e.g., aproportion of the destage/write activity 164 to the stage/read activity166 for the extent exceed a write-to-read proportion activity threshold,then control ends and the second cache caching setting 154 remainsdisabled until the extent is migrated back to the storage 10. If theproportion of read activity is sufficiently high, i.e., the proportionof write to reads is low, then the cache manager 24 performs (at block640) the operations in FIG. 19 to determine when to enable the secondcache caching 154.

Described embodiments provide techniques for managing caching operationswith respect to tracks in a second cache that caches tracks between afirst cache and a storage. In described embodiments, if an extent oftracks is migrated to the second cache from the storage, then thecaching of tracks in the extent from the first cache to the second cacheis disabled because the tracks of the extent are already maintained inthe first cache. Further, caching to the tracks in the extent may beenabled after a predetermined time or after the extent is migrated fromthe second cache back to the storage.

The described operations may be implemented as a method, apparatus orcomputer program product using standard programming and/or engineeringtechniques to produce software, firmware, hardware, or any combinationthereof. Accordingly, aspects of the embodiments may take the form of anentirely hardware embodiment, an entirely software embodiment (includingfirmware, resident software, micro-code, etc.) or an embodimentcombining software and hardware aspects that may all generally bereferred to herein as a “circuit,” “module” or “system.” Furthermore,aspects of the embodiments may take the form of a computer programproduct embodied in one or more computer readable medium(s) havingcomputer readable program code embodied thereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), an optical fiber,a portable compact disc read-only memory (CD-ROM), an optical storagedevice, a magnetic storage device, or any suitable combination of theforegoing. In the context of this document, a computer readable storagemedium may be any tangible medium that can contain or store a programfor use by or in connection with an instruction execution system,apparatus, or device.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmittedusing any appropriate medium, including but not limited to wireless,wireline, optical fiber cable, RF, etc., or any suitable combination ofthe foregoing.

Computer program code for carrying out operations for aspects of thepresent invention may be written in any combination of one or moreprogramming languages, including an object oriented programming languagesuch as Java, Smalltalk, C++ or the like and conventional proceduralprogramming languages, such as the “C” programming language or similarprogramming languages. The program code may execute entirely on theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider).

Aspects of the present invention are described above with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

The terms “an embodiment”, “embodiment”, “embodiments”, “theembodiment”, “the embodiments”, “one or more embodiments”, “someembodiments”, and “one embodiment” mean “one or more (but not all)embodiments of the present invention(s)” unless expressly specifiedotherwise.

The terms “including”, “comprising”, “having” and variations thereofmean “including but not limited to”, unless expressly specifiedotherwise.

The enumerated listing of items does not imply that any or all of theitems are mutually exclusive, unless expressly specified otherwise.

The terms “a”, “an” and “the” mean “one or more”, unless expresslyspecified otherwise.

Devices that are in communication with each other need not be incontinuous communication with each other, unless expressly specifiedotherwise. In addition, devices that are in communication with eachother may communicate directly or indirectly through one or moreintermediaries.

A description of an embodiment with several components in communicationwith each other does not imply that all such components are required. Onthe contrary a variety of optional components are described toillustrate the wide variety of possible embodiments of the presentinvention.

Further, although process steps, method steps, algorithms or the likemay be described in a sequential order, such processes, methods andalgorithms may be configured to work in alternate orders. In otherwords, any sequence or order of steps that may be described does notnecessarily indicate a requirement that the steps be performed in thatorder. The steps of processes described herein may be performed in anyorder practical. Further, some steps may be performed simultaneously.

When a single device or article is described herein, it will be readilyapparent that more than one device/article (whether or not theycooperate) may be used in place of a single device/article. Similarly,where more than one device or article is described herein (whether ornot they cooperate), it will be readily apparent that a singledevice/article may be used in place of the more than one device orarticle or a different number of devices/articles may be used instead ofthe shown number of devices or programs. The functionality and/or thefeatures of a device may be alternatively embodied by one or more otherdevices which are not explicitly described as having suchfunctionality/features. Thus, other embodiments of the present inventionneed not include the device itself.

The illustrated operations of figures showing events occurring in acertain order. In alternative embodiments, certain operations may beperformed in a different order, modified or removed. Moreover, steps maybe added to the above described logic and still conform to the describedembodiments. Further, operations described herein may occur sequentiallyor certain operations may be processed in parallel. Yet further,operations may be performed by a single processing unit or bydistributed processing units.

The foregoing description of various embodiments of the invention hasbeen presented for the purposes of illustration and description. It isnot intended to be exhaustive or to limit the invention to the preciseform disclosed. Many modifications and variations are possible in lightof the above teaching. It is intended that the scope of the invention belimited not by this detailed description, but rather by the claimsappended hereto. The above specification, examples and data provide acomplete description of the manufacture and use of the composition ofthe invention. Since many embodiments of the invention can be madewithout departing from the spirit and scope of the invention, theinvention resides in the claims herein after appended.

What is claimed is:
 1. A computer program product for managing data in afirst cache, a second cache, and a storage device, the computer programproduct consisting of a non-transitory computer readable storage mediumhaving computer readable program code embodied therein that executes toperform operations, the operations comprising: determining an eligibletrack in the first cache eligible for demotion to the second cache,wherein the tracks are stored in extents configured in the storagedevice, wherein each extent is comprised of a plurality of tracks;determining an extent including the eligible track; determining whethersecond cache caching for the determined extent is enabled or disabled;demoting the eligible track from the first cache to the second cache inresponse to determining that the second cache caching for the determinedextent is enabled; and selecting not to demote the eligible track inresponse to determining that the second cache caching for the determinedextent is disabled.
 2. The computer program product consisting of thenon-transitory computer readable storage medium of claim 1, wherein thefirst cache is a faster access device than the second cache and whereinthe second cache is a faster access device than the storage device. 3.The computer program product consisting of the non-transitory computerreadable storage medium of claim 2, wherein the first cache comprises atleast one dynamic random access memory (DRAM), the second cachecomprises at least one solid state storage device (SSD), and the storagedevice comprises at least one magnetic hard disk drive.
 4. The computerprogram product consisting of the non-transitory computer readablestorage medium of claim 1, wherein the operations of the determiningwhether the second cache caching for the determined extent is enabled ordisabled, the demoting the eligible track from the first cache, and theselecting not to demote the eligible track are performed when theeligible track includes unmodified data, wherein the operations furthercomprise: demoting the eligible tracks including modified data from thefirst cache to the second cache.
 5. The computer program productconsisting of the non-transitory computer readable storage medium ofclaim 1, wherein the operations further comprise: migrating an extentfrom the storage device to the second cache; and indicating the secondcache caching as disabled in response to the migrating of the extent tothe second cache.
 6. The computer program product consisting of thenon-transitory computer readable storage medium of claim 5, wherein theoperations further comprise: determining activity at the extents in thestorage device; and determining whether the activity for one of theextents exceeds a threshold level of activity, wherein the extent ismigrated from the storage device to the second cache in response todetermining that the determined activity for the extent exceeds thethreshold level of activity.
 7. The computer program product consistingof the non-transitory computer readable storage medium of claim 6,wherein the second cache stores only unmodified tracks demoted from thefirst cache, wherein the operations of the determining whether thesecond cache caching for the determined extent is enabled or disabled,the demoting the eligible track from the first cache, and the selectingnot to demote the eligible track are performed for the eligible tracksthat are unmodified, wherein the determining of the activity comprisesdetermining a write activity to the tracks in the extent and determininga read activity to the tracks in the extent, and wherein determiningwhether the activity for one of the extents exceeds the threshold levelof activity comprises determining whether at least one of the readactivity and the write activity exceeds the threshold level of activity:determining whether a proportion of the write activity to the readactivity for the extent exceeds a write/read proportion activitythreshold; and indicating that the second cache caching is enabled forthe extent after a predetermined time in response to determining thatthe proportion is less than the write/read proportion activitythreshold.
 8. The computer program product consisting of thenon-transitory computer readable storage medium of claim 6, wherein theoperations further comprise: gathering activity information for theextent for which the second cache caching is disabled; periodicallydetermining the activity for the extent while the extent is stored inthe second cache; determining whether the periodically determinedactivity is less than the threshold level of activity; and migrating theextent from the second cache to the storage device in response todetermining that the periodically determined activity is less than thethreshold level of activity.
 9. The computer program product consistingof the non-transitory computer readable storage medium of claim 8,wherein the operations further comprise: indicating the second cachecaching for the extent as enabled after a predetermined time from whenthe second cache caching for the extent was indicated as disabled; andindicating the second cache caching for the extent as disabled inresponse to determining that the periodically determined activity forthe extent is greater than the threshold level of activity.
 10. Thecomputer program product consisting of the non-transitory computerreadable storage medium of claim 8, wherein the operations furthercomprise: indicating the second cache caching for the extent as enabledin response to determining that the periodically determined activity isless than the threshold level of activity.
 11. The computer programproduct consisting of the non-transitory computer readable storagemedium of claim 6, wherein the determining of the activity comprisesdetermining a number of destage and stage operations from the firstcache or the second cache to the tracks in the extent in the storagedevice, wherein the threshold level of activity comprises a number ofdestage and stage operations to tracks in the extent in the storagedevice, and wherein the extent is migrated to the second cache inresponse to determining that the number of destage and stage operationsto tracks in the extent in the storage device exceeds the thresholdlevel of activity.
 12. The computer program product consisting of thenon-transitory computer readable storage medium of claim 6, wherein thedetermining of the activity comprises determining a distribution of thetracks in the extents located in the second cache or the storage devicesubject to write and read operations from the first cache, wherein thethreshold level of activity comprises a threshold value reflecting athreshold distribution of destage and stage operations to the extent inthe storage device, and wherein the extent is migrated to the secondcache in response to determining that the determined distributionexceeds the threshold value.
 13. A system coupled to a storage device,comprising: a processor; a first cache; a second cache; and a computerreadable storage medium having code executed by the processor to performoperations, the operations comprising: determining an eligible track inthe first cache eligible for demotion to the second cache, wherein thetracks are stored in extents configured in the storage device, whereineach extent is comprised of a plurality of tracks; determining an extentincluding the eligible track; determining whether second cache cachingfor the determined extent is enabled or disabled; demoting the eligibletrack from the first cache to the second cache in response todetermining that the second cache caching for the determined extent isenabled; and selecting not to demote the eligible track in response todetermining that the second cache caching for the determined extent isdisabled.
 14. The system of claim 13, wherein the first cache is afaster access device than the second cache and wherein the second cacheis a faster access device than the storage device.
 15. The system ofclaim 13, wherein the operations of the determining whether the secondcache caching for the determined extent is enabled or disabled, thedemoting the eligible track from the first cache, and the selecting notto demote the eligible track are performed when the eligible trackincludes unmodified data, wherein the operations further comprise:demoting the eligible track including modified data from the first cacheto the second cache.
 16. The system of claim 13, wherein the operationsfurther comprise: migrating an extent from the storage device to thesecond cache; and indicating the second cache caching as disabled inresponse to the migrating of the extent to the second cache.
 17. Thesystem of claim 16, wherein the operations further comprise: determiningactivity at the extents in the storage device; and determining whetherthe activity for one of the extents exceeds a threshold level ofactivity, wherein the extent is migrated from the storage device to thesecond cache in response to determining that the determined activity forthe extent exceeds the threshold level of activity.
 18. The system ofclaim 17, wherein the second cache stores only unmodified tracks demotedfrom the first cache, wherein the operations of the determining whetherthe second cache caching for the determined extent is enabled ordisabled, the demoting the eligible track from the first cache, and theselecting not to demote the eligible track are performed for theeligible tracks that are unmodified, wherein the determining of theactivity comprises determining a write activity to the tracks in theextent and determining a read activity to the tracks in the extent, andwherein determining whether the activity for one of the extents exceedsthe threshold level of activity comprises determining whether at leastone of the read activity and the write activity exceeds the thresholdlevel of activity: determining whether a proportion of the writeactivity to the read activity for the extent exceeds a write/readproportion activity threshold; and indicating that the second cachecaching is enabled for the extent after a predetermined time in responseto determining that the proportion is less than the write/readproportion activity threshold.
 19. The system of claim 17, wherein theoperations further comprise: gathering activity information for theextent for which the second cache caching is disabled; periodicallydetermining the activity for the extent while the extent is stored inthe second cache; determining whether the periodically determinedactivity is less than the threshold level of activity; and migrating theextent from the second cache to the storage device in response todetermining that the periodically determined activity is less than thethreshold level of activity.
 20. The system of claim 17, wherein thedetermining of the activity comprises determining a number of destageand stage operations from the first cache or the second cache to thetracks in the extent in the storage device, wherein the threshold levelof activity comprises a number of destage and stage operations to tracksin the extent in the storage device, and wherein the extent is migratedto the second cache in response to determining that the number ofdestage and stage operations to tracks in the extent in the storagedevice exceeds the threshold level of activity.
 21. The system of claim17, wherein the determining of the activity comprises determining adistribution of the tracks in the extents located in the second cache orthe storage device subject to write and read operations from the firstcache, wherein the threshold level of activity comprises a thresholdvalue reflecting a threshold distribution of destage and stageoperations to the extent in the storage device, and wherein the extentis migrated to the second cache in response to determining that thedetermined distribution exceeds the threshold value.
 22. The system ofclaim 14, wherein the first cache comprises at least one dynamic randomaccess memory (DRAM), the second cache comprises at least one solidstate storage device (SSD), and the storage device comprises at leastone magnetic hard disk drive.
 23. The system of claim 19, wherein theoperations further comprise: indicating the second cache caching for theextent as enabled after a predetermined time from when the second cachecaching for the extent was indicated as disabled; and indicating thesecond cache caching for the extent as disabled in response todetermining that the periodically determined activity for the extent isgreater than the threshold level of activity.
 24. The system of claim19, wherein the operations further comprise: indicating the second cachecaching for the extent as enabled in response to determining that theperiodically determined activity is less than the threshold level ofactivity.